The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Feb 25, 2026 0:26am

Greptile Overview

Greptile Summary

This PR adds two enhancements to the websearch interception handler: extracting API keys from router configuration and handling thinking parameter constraints for Anthropic models.

Key Changes:

• Extracts api_key and api_base from router's search_tools config and passes them to litellm.asearch()
• Adjusts max_tokens when it's less than or equal to budget_tokens by setting it to budget_tokens + DEFAULT_MAX_TOKENS (4096)
• Conditionally drops the thinking parameter when assistant messages with tool_calls have no thinking_blocks

Critical Issue Found:

• Lines 440 and 443 pass anthropic_messages (the imported module) instead of follow_up_messages (the messages list) to the helper functions. This will cause the thinking parameter logic to fail completely as it's analyzing the wrong data type.

Confidence Score: 1/5

• This PR contains a critical bug that will cause runtime failures
• The wrong variable (imported module instead of messages list) is passed to helper functions, causing the thinking parameter logic to fail. API key extraction looks correct but cannot be safely merged until the critical bug is fixed.
• litellm/integrations/websearch_interception/handler.py requires immediate attention - lines 439-444 must be fixed before merge

Sequence Diagram

sequenceDiagram
    participant Client
    participant WebSearchHandler
    participant Router
    participant LLM
    participant SearchProvider

    Client->>WebSearchHandler: Request with websearch tool
    WebSearchHandler->>WebSearchHandler: Pre-request hook: convert native tools
    WebSearchHandler->>LLM: Initial request
    LLM-->>WebSearchHandler: Response with tool_use blocks
    WebSearchHandler->>WebSearchHandler: Detect websearch tool_use
    
    Note over WebSearchHandler,Router: Extract API keys from router config
    WebSearchHandler->>Router: Get search_tools config
    Router-->>WebSearchHandler: search_provider, api_key, api_base
    
    loop For each search query
        WebSearchHandler->>SearchProvider: Execute search (with api_key)
        SearchProvider-->>WebSearchHandler: Search results
    end
    
    Note over WebSearchHandler: Check thinking parameter
    alt thinking.budget_tokens > max_tokens
        WebSearchHandler->>WebSearchHandler: Adjust max_tokens = budget_tokens + 4096
    end
    
    alt Last tool_call message has no thinking_blocks
        WebSearchHandler->>WebSearchHandler: Drop thinking parameter
    end
    
    WebSearchHandler->>LLM: Follow-up request with search results
    LLM-->>WebSearchHandler: Final response
    WebSearchHandler-->>Client: Return final response

Hey @Quentin-M, nice PR — both fixes address real production issues and the code is clean. A couple of things to address before merge:

1. Missing litellm.modify_params guard

The Anthropic and Bedrock transformations you're following both gate the thinking drop behind litellm.modify_params:

# litellm/llms/anthropic/chat/transformation.py:1114
if litellm.modify_params:
    optional_params.pop("thinking", None)

Your code drops thinking unconditionally. Users who have modify_params disabled would get unexpected behavior. The fix is straightforward — wrap the drop:

if should_drop_thinking:
    if litellm.modify_params:
        params_to_exclude.append('thinking')
        verbose_logger.warning(
            "WebSearchInterception: Dropping 'thinking' param because the last assistant message "
            "with tool_calls has no thinking_blocks. The model won't use extended thinking for this turn."
        )

2. Tests required

The project requires at least 1 test in tests/litellm/. Both the API key extraction logic and the thinking parameter handling are testable with mocks — e.g. mock llm_router.search_tools to verify api_key/api_base are extracted, and mock anthropic_messages_optional_request_params with various thinking/max_tokens combos.

Side note: Greptile flagged a "critical bug" claiming anthropic_messages (the module) was passed instead of follow_up_messages to the helper functions. That's incorrect — your code correctly passes follow_up_messages. Just wanted to flag that so you can disregard it.

@greptile

Greptile Overview

Greptile Summary

• Enables Bedrock + Claude extended/adaptive thinking to work with agentic tools by preserving/validating thinking blocks across the websearch interception loop.
• Centralizes Bedrock anthropic-beta whitelisting/translation with model version/family gating and uses it across Invoke Chat, Invoke Messages, and Converse.
• Strips context_management from Bedrock request bodies (header-only feature) and tweaks Router to expose resolved provider to callbacks.
• Adds Opus 4.6 adaptive thinking mappings and test coverage for thinking-block detection + beta header filtering.

Confidence Score: 2/5

• This PR has one clear merge-blocking correctness issue in Bedrock model ID constants; remaining changes look reasonable.
• Most changes are additive with tests (thinking block detection, beta header filtering, websearch interception follow-up), but the Opus 4.6 Bedrock Converse model ID in constants drops the ':0' suffix which will break routing/matching for that model until corrected.
• litellm/constants.py (BEDROCK_CONVERSE_MODELS Opus 4.6 entry)

Sequence Diagram

sequenceDiagram
    participant Client
    participant Router as litellm.Router
    participant WSI as WebSearchInterceptionLogger
    participant Provider as Bedrock/Anthropic
    participant Search as litellm.asearch

    Client->>Router: completion(model alias, messages, tools, thinking)
    Router->>WSI: pre_api_call(kwargs incl. resolved custom_llm_provider)
    WSI->>WSI: convert hosted web_search tool -> regular tool
    WSI-->>Router: return {**kwargs, tools: converted}
    Router->>Provider: initial LLM request
    Provider-->>Router: response(content blocks)
    Router->>WSI: async_post_call_success(response, kwargs)
    WSI->>WSI: transform_request() extracts tool_use + thinking blocks
    alt has websearch tool_use
        WSI->>Search: asearch(query, provider, api_key/api_base from router.search_tools)
        Search-->>WSI: search result(s)
        WSI->>WSI: transform_response() builds assistant msg (thinking + tool_use) + user tool_result
        WSI->>Provider: follow-up LLM request(max_tokens adjusted if <= thinking budget)
        Provider-->>WSI: final response
    else no websearch
        WSI-->>Router: passthrough
    end

Review Findings

Thanks for the comprehensive work on enabling websearch + extended thinking on Bedrock!

CI Status

Must Fix

1. Lint failure: Function too long

integrations/websearch_interception/handler.py:333:15: PLR0915 Too many statements (51 > 50)

The _execute_agentic_loop function now has 51 statements, exceeding Ruff's limit. Consider extracting some logic into helper functions (e.g., _validate_max_tokens_for_thinking()).

2. Multiple test failures in beta headers code

Several new tests are failing:

• test_context_management_requires_claude_4_5
• test_backward_compatibility_existing_headers
• test_converse_anthropic_model_gets_anthropic_beta
• test_advanced_tool_use_header_translation_for_opus_4_5
• test_converse_filters_unsupported_headers
• And 10+ more...

The beta header filtering logic appears to have bugs in version/model matching that need investigation.

3. Missing litellm.modify_params guard

The existing Anthropic and Bedrock transformations gate thinking drop behind litellm.modify_params:

# Anthropic transformation.py:1114
if litellm.modify_params:
    optional_params.pop("thinking", None)

The PR drops thinking unconditionally without checking this flag.

Positive Aspects

1. Centralized beta header filtering (beta_headers_config.py) is a good architectural improvement
2. TransformRequestResult NamedTuple makes the transform_request contract explicit
3. Comprehensive test coverage - 2,400+ lines of new tests
4. Good documentation - New README.md in litellm/llms/bedrock/
5. Real user-facing bugs fixed - Bedrock/Claude with websearch + thinking

Suggestion

Consider waiting for upstream PRs (#20488, #20514, #20519) to merge first, then rebasing. This would reduce the PR size and avoid potential merge conflicts with poetry.lock.

Greptile Summary

This PR implements three main feature areas: (1) websearch interception with extended thinking support on Bedrock, (2) centralized beta header filtering for all Bedrock APIs, and (3) improved thinking param handling for assistant messages without thinking blocks.

Key concerns:

• Removes native structured outputs for Bedrock Converse — _supports_native_structured_outputs(), _create_output_config_for_response_format(), _add_additional_properties_to_schema(), and outputConfig handling were all deleted. This removes production functionality for Claude 4.5+, Qwen3, Mistral, and DeepSeek models. Multiple existing tests (at least 5) will break.
• Renames _is_nova_2_model to _is_nova_lite_2_model — narrows matching to exclude Nova-2-Pro, which is a listed model with reasoning support. Existing tests call the old method name and will fail with AttributeError. Nova-2-Pro will no longer receive correct reasoningConfig parameters.
• Removes Sonnet 4.6 from interleaved thinking and tool search support — patterns for Sonnet 4.6 were stripped from _supports_interleaved_thinking_on_bedrock() and _supports_tool_search_on_bedrock().

What works well:

• The centralized BedrockBetaHeaderFilter in beta_headers_config.py is well-designed with version-based filtering, family restrictions, and header translation
• Thinking block preservation through the websearch agentic loop is thoroughly implemented and tested
• Loading api_key/api_base from router's search_tools config fixes the "TAVILY_API_KEY is not set" issue
• New tests are comprehensive and mock-based (no real network calls)
• The max_tokens auto-adjustment when <= thinking.budget_tokens follows existing patterns

Confidence Score: 2/5

• This PR introduces regressions by removing native structured outputs and Nova-2-Pro reasoning support, which will break existing tests and functionality.
• While the websearch interception and beta header centralization changes are well-implemented and well-tested, the converse_transformation.py changes silently remove production features (native structured outputs, Nova-2-Pro reasoning) that have existing test coverage. These removals will cause test failures and functional regressions for users relying on those features.
• litellm/llms/bedrock/chat/converse_transformation.py requires immediate attention — it removes native structured outputs and Nova-2-Pro reasoning support. litellm/llms/bedrock/messages/invoke_transformations/anthropic_claude3_transformation.py needs verification of Sonnet 4.6 pattern removals.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User Request with web_search tool] --> B{WebSearchInterceptionLogger\nasync_log_pre_api_call}
    B -->|Provider not enabled| C[Pass through to LLM]
    B -->|Provider enabled| D[Convert web_search to\nLiteLLM standard tool]
    D --> E[LLM Response]
    E --> F{async_should_run_agentic_loop}
    F -->|No WebSearch tool_use| G[Return response]
    F -->|WebSearch tool_use detected| H[Extract tool_calls +\nthinking_blocks]
    H --> I[_execute_agentic_loop]
    I --> J[Load search credentials\nfrom router search_tools]
    J --> K[Execute parallel searches\nvia litellm.asearch]
    K --> L[Build follow-up messages\nwith thinking + tool_result]
    L --> M{max_tokens <= budget_tokens?}
    M -->|Yes| N[Adjust max_tokens =\nbudget + DEFAULT_MAX_TOKENS]
    M -->|No| O[Keep original max_tokens]
    N --> P[anthropic_messages.acreate\nfollow-up request]
    O --> P
    P --> Q[Return final response]

    subgraph Beta Header Filtering
        R[anthropic-beta headers] --> S{BedrockBetaHeaderFilter}
        S --> T[Whitelist check]
        T --> U[Version-based filtering]
        U --> V[Family restrictions]
        V --> W[Header translation\ne.g. advanced-tool-use]
        W --> X[Filtered headers to AWS]
    end

_{Last reviewed commit: ab743af}

All committers have signed the CLA.

@ryangoldblatt-bm to sign the CLAs 🙏

@ryangoldblatt-bm to sign the CLAs 🙏

Done sir, shall we reopen this?